Autonomous vehicles, such as vehicles which do not require a human driver, may be used to aid in the transport of passengers or items from one location to another. An important component of autonomous vehicles is the perception system, which allows the vehicle to perceive and interpret its surroundings using cameras, different types of sensors, and global positioning devices. The perception system executes numerous decisions while an autonomous vehicle is in motion, such as speeding up, slowing down, stopping, turning, etc. Autonomous vehicles may also use the cameras, sensors, and global positioning devices to gather images and sensor data about its surrounding environment, e.g., roads, buildings, sidewalks, etc. These images and sensor data allow the vehicle to safely maneuver itself around obstacles. In addition, this information may be examined in an offline environment in order to improve the vehicle's object detection and maneuvering capabilities.